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Abstract 

In the Randall- Sundrum model, the radion-Higgs mixing is weakly suppressed by the effective 
electroweak scale. A novel feature of the gravity-scalar mixing would be a sizable three-point 

(n) 

vertex of h^J-h-cj). We explore the potential of photon colliders, achieved by the laser backscattering 
technique, in probing the radion-Higgs mixing via the associated production of the radion with the 
Higgs boson. The advantage of photon colliders is the capability of adjusting the polarization of the 
incoming photons such that the signal of the spin-2 graviton exchange can be largely enhanced. 
The enhancement factor is shown to be about 5, except for small-£ region. We also study the 
corresponding backgrounds step-by-step in detail. 
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I. INTRODUCTION 



The standard model (SM) has been extraordinarily successful up to now in explaining all 
experimental data on the electroweak interactions of the gauge bosons and fermions. How- 
ever, the master piece of the SM, the Higgs boson, which is responsible for the electroweak 
symmetry breaking, still awaits experimental discovery [lj]. The direct search has excluded 
the SM Higgs boson mass below about 114 GeV 21, while the indirect evidences from the 

n 

electroweak precision data put an upper bound for a SM Higgs of 208 GeV at 95% C.L. |3|]. 
The precision measurements and the direct searches are getting into the situation that they 
begin to contradict each other. There have been various studies which can ease the situation. 
One possibility is that the SM Higgs boson mixes with another scalar boson such that the 
Higgs branching ratio into bb becomes smaller and thus escapes the limit of direct search so 
far. Disentangling the nature of this new scalar state is very significant and challenging. 

It has been pointed out that the radion of the Randall and Sundrum (RS) model ^| 
can play the role of such a scalar boson. The RS model consists of an additional spatial 
dimension of a S 1 /Z 2 orbifold introduced with two 3-branes at the fixed points. A geo- 
metrical suppression factor, called the warp factor, explains the huge hierarchy between the 
electroweak and Planck scale with moderate values of model parameters. The presence of a 
radion, the quantum degree with respect to the fluctuation of the brane separation, naturally 
emerges from the stabilizing process 0, f|. As various stabilization mechanisms suggest, the 
radion is generically much lighter than the Kaluza-Klein (KK) states of any bulk field. In 
the literature, phenomenological aspects of the radion have been studied such as its decay 
modes 0, [J, its effects on the electroweak precision observations and its signatures at 
present and future colliders jl(| . 

The radion-Higgs mixing is originated from the gravity-scalar mixing term, ^R{g vls )W H ^ 
where R(g V is) is the Ricci scalar of the induced metric g^ s , and H is the Higgs field in the five- 
dimensional context. It has been shown that the radion-Hi ggs m ixing can induce significant 
deviations to the properties of the SM Higgs boson . A complementary 

way to probe the radion-Higgs mixing is the direct search for the new couplings exclusively 
allowed with a non-zero mixing parameter £. One good example is the tri-linear vertex 



among the KK graviton field hffi, the Higgs boson h, and the radion <p. In Refs. 16l ll7|. we 



have shown that probing the vertex h^J-h-cj) through the h(j) production at e + e~ colliders 
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and hadronic colliders can provide very useful information on the radion-Higgs mixing, 
irrespective of the mass spectrum of the Higgs boson and the radion. 

In this work, we turn to photon colliders achieved by the laser backscattering technique 
Q|. The process that we investigate is 

77 - &<S -> h<j> , (1) 

where hffi denotes the n-th KK state of the RS graviton. Since the polarization of incoming 
photons can be adjusted by tuning the polarization of the electron or positron beam and the 
laser beam, the signal can be largely enhanced because the exchanged graviton is a spin-2 
particle. This is the biggest advantage of photon colliders in this regard. The observation 
of the rare decay of a KK graviton into h(j> is then the direct and exclusive signal of the 
radion-Higgs mixing. In addition, the characteristic angular distribution could reveal the 
exchange of massive spin-2 KK gravitons. 

This paper is organized as follows. In Section II, we calculate the production cross section 
of 77 — > hffi —> h(fi folded with the photon luminosity function. Section III deals with the 
feasibility of detecting the hip final states by considering specific decay channels of the Higgs 
boson and the radion. We summarize at the end of Section III. Note that we shall use 

G (n) 

or h^J to denote the n-th KK graviton state interchangeably. 

II. CALCULATION OF 77 -> h$ -► h<f> 

The RS scenario is based on a five-dimensional spacetime of a S 1 /Z2 orbifold which has the 
finite size of b . The warped factor, fi — e~ m ° b °/ 2 , with a moderate value of m b /2 ~ 35 
can solve the gauge hierarchy problem. In terms of the KK graviton field hfy) and the 
canonically normalized radion field <fio, the four-dimensional effective Lagrangian is then 

X 1 00 

c = -pr; - ^t^(x) £ h$(x) , (2) 

A </> AW n=l 

where A^, is the vacuum expectation value (VEV) of the radion field, is the trace of the 
symmetric energy-momentum tensor T Miy , and Aw is the effective electroweak scale. Both 
effective interactions are suppressed by the electroweak scale, not by the Planck scale. The 
gravity-scalar mixing term of = £ / d 4 Xy/g v - m R(g w is)H^ H 0, [3| is allowed as it respects 
all the SM symmetries and Poincare invariance. Here g v ; s is the induced metric on the visible 
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brane, R(g vls ) is the Ricci scalar, Hq = QqH, and the dimensionless parameter £ of order 
one denotes the size of the mixin g te rm. This £-term mixes the ho and 0o fields into the 



mass eigenstates of h and fields 



n 




d c 
b a 



( 



(3) 



We refer the detailed expressions for a, b, c, and d to Ref. |17l |. 

All phenomenological signatures of the RS model are then determined by five parameters 

of 

?, A , — — , m h m h , (4) 
which in turn determine Aw = A^/^/3 and KK graviton masses m G ( n ) = x n moAw / (Mp\\/2) 



re first kind. The ratio 
I9I ] . In what follows, we 



with x n being the n-th root of the first order Bessel function of t 
mo/Mpi is assumed in [0.01, 0.1] to avoid too large bulk curvature 
fix the ratio mo/Mpi = 0.1. The or Aw is constrained by the Tevatron Run I data of 
Drell-Yan process and by the electroweak precision data: For m /Mp\ = 0.1, itlq > 500 
GeV yields A^ > 3.2 TeV j^. Therefore, we consider the case of A^ = 3.5 TeV and 
mo/Mpi = 0.1, of which the effect of radion on the oblique parameters is small 9}. Then, the 
first KK graviton mass is about 547.5 GeV. The radion mass is expected to be light as one 
of the simplest stabilization mechanisms predicts m^ ~ A^y/40 [5j. In addition, the Higgs 
boson mass is set to be 120 GeV through out the paper. 

The gravity-scalar mixing modifies the couplings among the h, <p and . In particular, 
a non-zero £ gives rise to new tri-linear vertices of 

h$-h-<j>, h$-<l>-<f>, h-(f>-4>, 0-0-0. (5) 

Due to the suppressed coupling of a photon with a Higgs boson or a radion, the 77 collider is 
expected to access the h^J-h-cf) or h^J-(j)-(j) vertex directly. In addition, the coupling strength 
of Zi^-0-0 is much smaller than that of hffl-h-(j), by a factor of 7 = v/A^ <S 1. Here v is the 
VEV of the Higgs boson, which is 246 GeV. Therefore, the channel h^J — > hep is the most 
effective in probing the radion-Higgs mixing, with the vertex denoted by 

(^1^10)-^%^, (6) 

Aw 



-A- 




FIG. 1: Feynman diagrams for the process 77 



where gch<t> — 67^ [0(76 + d) + be] +cd and k 12 is the four-momentum of the scalar particles. 
Then, the partial decay width of hffi — > h<p is given as 

~2 3 -. 2 ° 



w 



(7) 



where Ht V = (m x /m y ) 2 , (3 = A 1 / 2 (l, fi^c, H-hc), and A(a, 6, c) = a 2 + b 2 + c 2 — 2ab — 2ac — 2bc. 
In Ref. |l7j ]. it was shown that the branching ratio Br(/i^/ — > h<fi, (fxfi), which would vanish 
in the limit £ — > 0, is of the order of O(10~ 3 ). 
For the process of 

l(Qi, AO + T(?2: *a) -> fc(fci) + 0(A?a) , (8) 

the Feynman diagrams are shown in Fig. ^ Here Ai^ is the polarization of the high energy 
photons. The helicity amplitudes Ai\ 1 \ 2 including the h and <fi mediation are 



M ++ 
M+- 



2vA 

s 



A(l, m l h l §, mij s)g G<t>h V G sin 2 6* 



A 2 
1)l w 



(9) 
(10) 



where s = (qi + 92) , 6* is the scattering angle of the Higgs boson in the 77 cm. frame, c 77 h 
and C'yy^j are 



C77/1 



a 
2k 
a 
2~ix 



(d + 76) £ e 2 KF t (4m 2 /m 2 h ) + ^b 
(c + 7 a) £ elNiF^ml/ml) + y 7 « 



(11) 



We refer the expressions for Fx/2, -F, and to Ref. The propagator factors of the 
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KK-graviton, the Higgs boson, and the radion are given by 

s 



V, 



G 



E 

71=1 



m 



V 



h,4> 



m 



h,<t> 



(12) 



Note that M.++ = M. and M.+- = M. y are guaranteed by CP invariance. In principle, 

the photon polarization can separate the contribution of the scalar mediation from that of 
KK gravitons even though leading contribution comes from the KK graviton mediation. 

Brief comments on 77 colliders are in order here From the head-on collisions between 
the laser and energetic electron (or positron) beams, high energy photons are produced. If 
we denote the fraction of the photon beam energy to the initial electron beam energy by 
x = E 1 /E e , its maximum value is x max = z/(l + z) where z = AE e ujQ/m 2 e . Here E 1 , E e , uoq 
are the photon, electron and laser beam energies, respectively. Usually, z is optimized to be 
2(1 + v2) to avoid the e + e~ pair production through the interactions of the laser beam and 
the backward scattered photon beam. In the numerical analysis, we consider the following 
range for X\ 2' 



\z=2(l+V2) ■ 



(13) 



V0.4 < xi( 2 ) < x x _ 

With the given polarizations of the laser and parent electron (positron) beams, their 
Compton back-scattering leads to the differential cross section 
do 



d cos 9* 



-J- / I d Xl dxJ Mf ^ ] fl,^,^ 
32ns ee J J X1X2 \ s s , 



1 + 6(^1)6(^2; 



M 



Jz=0 



2 a 
+ 



6(^1)6(^2) 



M 



Jz=2 



(14) 



where s ee = s/(x\X2) is the square of the cm. energy of the parent e + e collision, and 
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(15) 



Here f(x) is the photon luminosity function and £2 0*0 is t ne averaged circular polarization of 
the back-scattered photon beam, both of which depend on the polarizations of the electron 
P e and laser beam P;. The explicit expressions for f(x) and 6( :c ) are 



where 



Or 



4 

z 



f{x,P e ,P l ;z) = —C{x) 



hx{z + 1) + - + - 



2{z + l)< 



(16) 



(17) 
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FIG. 2: Total cross section for 77 — > h<j) in fb as a function of £ for = 30, 70, 150 GeV at 
e + e~ colliders running in the 77 mode using laser backscattering. All of the polarizations are set 
to be zero, m h = 120 GeV, A,* = 3.5 TeV, and Js^ e = 500 GeV. 



PePl 



1 + - \n(z +1) 1 ; — 

z J K 1 2 z+1 2z+l 2 



and 



C{x) 



x 



(1 



X 



4 r (l - r ) - P e P ir z(2r - l)(2-x), 



where r = x/[z(l — x)]. The average helicity £2(2^ Pei Pi] z ) is given by 



Ux,Pe,Pf,z) 



C(x) 



X 



X 



+ x{2r - V 



- Pi (2r - 1) ( 1 - x + 



1 



x 



(18) 



(19) 



In Fig. El we plot the total cross section as a function of £ for = 30, 70, 150 GeV. 
We set rrih = 120 GeV, A^ = 3.5 TeV and ^fs^ e = 500 GeV. Note that the requirement of 
positive definiteness of the mass and kinetic terms limits the range of the mixing parameter 
£. Here we assume that all beams are unpolarized. For = 30 GeV, the maximum 
total cross section can reach about 1 fb, which will produce about 1000 events with 1 ab _1 
luminosity. For heavier radion mass, the more restricted £-range reduces the maximum of 
the total cross section, e.g., to several 10~ 2 fb for = 150 GeV. 

Since the 77 — > hcf) process is practically mediated by the massive spin-2 KK graviton, 
the angular distribution shows its characteristic behavior proportional to sin 4 #*, as shown 
in Eq. (|l()jl. In Fig. 01 we plot the normalized differential cross section (1 / a) da jd cos 6* 
versus cos^* for = 30, 70, 150 GeV. Here we have set = 3.5 TeV, and £ = 0.26 
which is the allowed maximum value for = 150 GeV. For = 30 GeV and = 70 
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-1 -0.5 0.5 1 

cos 9* 

FIG. 3: Normalized differential cross section d for = 30, 70, 150 GeV. The 
dependence is mild. All of the polarizations are set to be zero, mh = 120 GeV, = 3.5 TeV and 
f = 0.26. 

GeV, the distributions are practically the same, proportional to sin 4 #*. For = 150 GeV, 
the enhanced contributions from the Higgs boson and radion alter the behavior slightly. 
Especially, at the end points (cos#* = ±1) the cross section comes solely from the Higgs 
and radion exchanges. One crucial reason is that the heavy radion mass reduces the KK- 
graviton contribution which has an additional factor of A 2 (l, m\/s, m^/s) with respect to 
the h/<f> contribution, as can be seen from Eqs. © and l|10|) . 

As the general expression of Eq. ((H}) suggests, the appropriate adjustment of the electron 
(or positron) and laser beam polarizations can enhance the production rate. As can be seen 
from Eq. (|14j) . non-zero and negative ^2(^1)^2(^2) can enhance the graviton contribution. 
Another merit is that the polarized beams can enhance the energy of back-scattered photon 
beam and thus increase the signal cross section. Figure H] presents f(x)/x where f(x) = 
f{x)£,2{x) for the polarized beam while f(x) = f(x) for the unpolarized beam 1 . Compared to 
the unpolarized beam case, (P e , Pi) = (1, —1) combination generates a more energetic photon 
beam. Moreover, the spin-2 nature of KK graviton prefers opposite polarizations for e + and 
er beams. Therefore, the optimal polarizations would be (P e -, P e +, P11, P12) — (1> — 1> — 1) !)• 

With the optimal polarization combination, we plot the ratio of the polarized total cross 
section to unpolarized one, as a function of £ for = 30, 70, 150 GeV in Fig. |S| The £- 

1 The average helicity function becomes zero for P e = Pi = 0. 
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FIG. 4: f(x)/x versus x for various polarizations of the electron and laser beams, denoted by 
(P e ,Pi). The f(x) = f(x) for unpolarized beams (P e = Pi = 0), and f(x) = f(x)^2(x) for 
polarized beams with P e = 1. The beam polarizations of (P e ,P[) = (1, —1) generate the highest 
energy photon beam from the parent electron beam. 
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FIG. 5: The ratio of the polarized total cross section to unpolarized one, as a function of ^ for 
rricf, = 30, 70, 150 GeV. The polarization is set to be (P e - , P e + , Pn, P12) = (1, —1, —1, 1), rrih = 120 
GeV, JJ^ e = 500 GeV, and A,* = 3.5 TeV. 



dependence is negligible in the region of |£| > 0.3. The enhancement is significant and better 
than the naive estimation of a factor of two from Eq. (|14|) . The enhancement factor can 
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reach upto 5.1 for = 150 GeV, and about 4.8 for = 30, 70 GeV. Even for the = 70 
GeV case, the total cross section can be about 1 fb. Under conservative assumption for the 
electron and positron beam polarization, we set (P e -, P e +, Pa, Pn) — (0-8, —0.6, —1, 1): the 
maximum of the enhancement factor is about 3.3 for = 30, 70 GeV, and about 3.6 for 
= 150 GeV. 

III. DECAYS AND DETECTION OF THE RADION-HIGGS PAIR 

In this section, we consider the feasibility of detecting h(j) pair production. For a Higgs 
boson of mass around 120 GeV, the major decay mode is into bb. The partial decay rate 
into WW will begin to grow at > 140 GeV. Therefore, we shall focus on the bb mode 
for the Higgs boson decay. A light radion, on the other hand, has the major decay mode of 
gg because of the QCD trace anomaly, followed by bb (a distant second). When the radion 
mass gets above the WW threshold, the WW mode becomes dominant. 

The major background comes from the QCD heavy-flavor production of 

77 -> bb/cc + 2 jets , (20) 

where each jet can come from a gluon or a light quark. Here the cc pair can also fake the B- 
tag though with a much lower probability than the b quark. We calculate the QCD bb/cc + 2 
jets background by a parton-level calculation, in which the sub-processes are generated by 
MADGRAPH (2j|. Another possible source of background is 

77 -> W + W- , (21) 

followed by the hadronic decay of the WW pair. The decay of the W boson into a b quark is 
severely suppressed by (V c b + V u b) 2 — (0.05) 2 . The chance of seeing two b quarks in the WW 
decay is very rare, of the order of 6 x 10~ 6 . On the other hand, WW production is still a 
possible background because of the W boson decay into a c quark. The charm quark may be 
tagged with a displaced vertex with a small mistag probability, thus may be misinterpreted 
as a b quark. 

We assume a 50% S-tagging efficiency and a chance of 5% mistag (a charm quark misin- 
terpreted as a 6 quark) in our study. Typical cuts on detecting the 6-jets and light jets are 
applied: 

p T (b) > 15 GeV, p T (j) > 15 GeV, 



I cos^| < 0.9, | cos^'l < 0.9 , cos0 6> 5, cos 9jj, cosfl&j < 0.9, 

where cos 6^ and cos#j denote the cosine of the angle of the outgoing b quark and the jet, 
respectively, and cos 8 b i, cos 9jj, and cos 0&j denote the cosine of the angle between the two b 
quarks, between the two jets, and between the b quark and the jet, respectively. The angular 
cuts are mainly for the detection purpose and for removing the collinear divergence in the 
calculation. We have applied a gaussian smearing AE/E = 0.5/VE, where E is in GeV, to 
the final-state 6-jets and light jets to simulate the detector resolution. Since the Higgs boson 
is produced together with a radion mainly via an intermediate graviton KK state, the Higgs 
boson tends to have a large pt ~ m G (i)/2. Therefore, a transverse momentum cut on the 
bb pair is very efficient against the QCD background while only hurts the signal marginally. 
We apply a cut 

p T (bb) > 100 GeV , (22) 

to reduce the background. Finally, we apply the invariant mass constraint of the bb pair to 
be near the Higgs boson mass and that of the jet pair to be near the radion mass: 

\m b i — ttihI < 10 GeV , \ m jj ~ m <f>\ < 10 GeV . (23) 

We summarize the cross sections for the signal and various backgrounds under successive 
cuts in Table H] The final signal-to-background ratio is quite promising. For = 30 GeV 
and £ = 1.4 we obtain a signal-to-background ratio about 1.3 : 1. For = 70 GeV with 
£ = 0.6 a ratio of 1.4 : 1 can be obtained. A ratio of 0.37 : 1 is obtained for the case of 
= 150 GeV and £ = 0.25. Note that the signal cross section scales as £ 2 as long as the 
positive mass-square constraints for the scalar bosons are satisfied. 

In Table |H] we show the results at ^/s^ : = 0.8 TeV, at which the effect of the first 
graviton resonance is large, such that the signal cross section is substantially larger than 
the background after the cuts. This is because the majority of the photon collisions are 
at Vs = ^xiX2-s ee ~ (0.7 — 0.8)(800 GeV), which is very close to the first KK graviton 
resonance. 

To disentangle possible conflict between the electroweak precision data and the direct 
search bound of the SM Higgs, we studied the possibility that the SM Higgs mixes with the 
radion of the RS model such that the Higgs branching ratio into bb becomes smaller and thus 
escapes the limit of direct search. We have explored the direct search of the radion-Higgs 
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TABLE I: Cross sections in fb under successive application of the cuts mentioned in the text at 
y/s^ = 0.5 TeV. Unpolarized beams are chosen. The results for = 30 (£ = 1.4), 70 (£ = 0.6) 
and 150 (£ = 0.25) GeV are shown in the corresponding rows. 



Cuts 


h(j) signal 


bbjj 


ccjj 


W+W- -► ccjj 


p T (b,j) > 15 GeV 


0.14 








|cos0 bj6j | < 0.9 


(0.16) 


39 


346 


287 


cos 9 bij < 0.9 


(0.018) 








additional 


0.14 








p T (bb) > 100 GeV 


(0.15) 


4.5 


38 


26 




(0.016) 








additional 


0.14 


0.092 


1.32 


0.0009 


\M bl -m H \ < 10 GeV 


(0.15) 


(0.091) 


(1.0) 


(0.242) 


IMjj-m^l < 10 GeV 


(0.014) 


(0.027) 


(0.215) 


(0.82) 


with B-tag e b = 0.5 


0.034 


0.023 


0.0033 


2 x 10~ 6 


or C-mistag e c = 0.05 


(0.037) 


(0.023) 


(0.0025) 


(6 x KT 4 ) 




(0.0035) 


(0.007) 


(5 x 10- 4 ) 


(2 x 10~ 3 ) 



associated production at photon colliders, and shown that the photon colliders achieved by 
the laser backscattering technique are very special in probing such a process. As is well 
known, the advantage of photon colliders is the capability of adjusting the polarization 
of incoming photons such that the signal of the spin-2 graviton exchange can be largely 
enhanced compared to the unpolarized collision. We have found the enhancement factor is 
around five, unless £ is quite small. This can be attained by aligning the polarization of the 
incoming photons such that the nature of the spin-2 graviton exchange is fully enhanced. We 
have also studied the corresponding background to h<f> pair productions at photon colliders 
step-by-step in detail. Specifically, we have considered the major backgrounds from the 
QCD heavy flavor production of 77 — > bb/cc + 2 jets and 77 — > W + W~ — > cc + 2 jets. By 
imposing the various cuts, we have shown that the associated production signal of h(p can 
be comparable or even much larger than those backgrounds at photon colliders. 
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TABLE II: Cross sections in fb under successive application of the cuts mentioned in the text at 



y/s^ = 0.8 TeV. Unpolarized beams are chosen. The mixing parameter £ = 1.4 and = 30 GeV 
are chosen. 





/i0 signal 


bbjj 


ccjj 


KK — > CCjj 


Pt{o, J ) > io v^e v 










1 cos0 b,b,j\ < °- 9 


b.zo 






1DO 


COS 1/5^ \ U.c/ 










additional 










PT {bb) > 200 GeV 


6.10 


0.86 


7.9 


3.0 


additional 










\M bl -m H \ < 10 GeV 










\M fj -m4,\ < 10 GeV 


5.95 


0.0015 


0.031 


~ 


with B-tag eh = 0.5 










or C-mistag e c = 0.05 


1.49 


0.0004 


8 x 10~ 5 


~ 
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